Systran Mt Dictionary Development

SYSTRAN has demonstrated success in the MT field with its long history spanning nearly 30 years. As a general-purpose fully automatic MT system, SYSTRAN employs a transfer approach. Among its several components, large, carefully encoded, high-quality dictionaries are critical to SYSTRAN's translation capability. A total of over 2.4 million words and expressions are now encoded in the dictionaries for twelve source language systems (30 language pairs one per year!). SYSTRAN'S dictionaries, along with its parsers, transfer modules, and generators, have been tested on huge amounts of text, and contain large terminology databases covering various domains and detailed linguistic rules. Using these resources, SYSTRAN MT systems have successfully served practical translation needs for nearly 30 years, and built a reputation in the MT world for their large, mature dictionaries. This paper describes various aspects of SYSTRAN MT dictionary development as an important part of the development and refinement of SYSTRAN MT systems. There are 4 major sections: 1) Role and Importance of Dictionaries in the SYSTRAN Paradigm describes the importance of coverage and depth in the dictionaries; 2) Dictionary Structure discusses the specifics of dictionary structure and types of information represented; 3) Dictionary Creation and Update describes the strategy and mechanics of the dictionary development; 4) Past. Present and Future Development provides some perspective on where SYSTRAN has come from and where it is going. 1. The Role and Importance of Dictionaries in the SYSTRAN Paradigm The problem of automatic linguistic analysis is one of accumulating and utilizing linguistic knowledge. Much of the needed knowledge is static and can be built into the system and its dictionaries, e.g. standard syntax, and conventional word use, including the variations typical of different text types and domains. MT systems encode knowledge in a variety of ways. Linguistic rules, probabilities, and databases of examples are typical encoding methods. SYSTRAN, which is a rule-based transfer system, encodes linguistic knowledge primarily in two ways: 1) In the dictionaries which contain “bottom-up” parse rules along with extensive syntactic and semantic information about each word; 2) In linguistic programs which contain the general “top-down” rules in the parser. Dictionary coverage, or size, is critical to high-quality translation. Words not found in a system's dictionary are a challenge to parse correctly. For SYSTRAN, broad coverage has always been a high priority because of the extensive use of SYSTRAN for information gathering in a wide variety of text types and domains (Gachot 1996.)